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ABSTRACT 

For the exploitation of the ERTS-1 images, both multispectral and spatial 
pattern recognition techniques are important. A multispectral recognition 
software program was successfully tested. A technique for the generation of 
resource boundaries has been successfully combined with terrain recognition 
software resulting in a regionalized terrain classification system. Digital 
registration of multispectral images has been achieved. 

Also, photointerpretation results from one test site are reported and the 
development of spatial terrain signatures from optical diffraction patterns is 
described. 
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PREFACE 


a. Objective 

The objective of the investigation is to develop digital interpretation 
techniques such that earth resources and their changes with time can be re- 
cognized automatically from ERTS-1 images. Such techniques are expected to 
increase the utility of the large volume of imagery produced by the satellite 
by providing for efficient processing of the data. 

b. Scope of Work 

Under the investigation, multi-spectral and spatial pattern recognition 
techniques . are being developed so that both the multi-spectral and spatial 
pattern information in the ERTS-1 images can be exploited. Interpretation 
results from both methods will be merged to expand the number of resources 
that can be recognized and increase the accuracy of recognition. 

ERTS images from six test sites will be subjected to the interpretation 
process to provide a variety of resources for testing the techniques . 

c. Conclusions 

The research activities during the period covered by the report indicate 
that the objective of the investigation can be met. Spatial signatures can 
be used to recognize terrain types; boundaries enclosing resource classes 
can be generated and multi -spectral programs can recognize a different group 
of resources. Furthermore, through digital image registration, it is poss- 
ible to digitally determine changes in resources with time. 

d . Recommendations 

The digital processing is complex and best results are achieved when the 
investigator can visually monitor the outputs of the processing operations. 
Therefore, it is recommended that man-computer interaction be increased 
through computer driven visual displays. 
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1 . INTRODUCTION 


This is the Type II report required by the contract for the period of 
June 15 - November 15, 1972. The purpose of the report is to describe the 
technical progress of the project for this period. 

The main body of the report is divided into the following sections: 

A. Input Data Operations, which describes the data that has been re- 
ceived from NASA, the data handling operations necessary for the utilization 
of the data, and the data required to complete the project. 

B. Photointerpretation Tasks complete and in progress. These tasks 
are necessary for the location of ground truth, familiarization with the 
test sites and the ERTS images and for evaluating the digital processing 
operations . 

C. Digital Interpretation Techniques. This section describes develop- 
ment of software packages that are essential to the digital processing of 
ERTS images. This section includes a multi-spectral detection program 
which recognizes resources by their reflectances in the ERTS spectral bands, 
a program that develops boundaries enclosing areas occupied by resources of 
the same classification, and a program which registers digital images of the 
same or different spectral bands. This program is essential for performing 
change detection between ERTS images of a specific area obtained at differ- 
ent times. 

D. Development of Terrain Spatial Signatures. This section describes 

a task for developing spatial signatures for various types of terrain. These 
can then be used with our spatial detection program to recognize terrain 
types. The signatures are developed optically from the diffraction patterns 
of small areas of the ERTS images and their detection is then implemented 
digitally. 

E. Conclusions. This section describes the conclusions that can be 
drawn from the work performed. 

F. Program for Next Reporting Interval. The work planned for the re- 
maining time interval of the investigation is presented in this section. 

G. Recommendations. 
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The intent of this report is to provide a clear understanding of the 
work performed. However, significant results and technical details from 
the above tasks will be reported in separate reports. 

2. INPUT DATA OPERATIONS 

2.1 ERTS Images 

Since the delivery of the first ERTS image on September 12, 1972, a sub- 
stantial number of 9-1/2 inch positive images has been delivered to the pro- 
ject fulfilling one of the goals of the ERTS program for repetitive complete 
coverage of the continental United States and many other parts of the world. 
Most of the images arrived during October and November 1972. A system of 
checking and cataloging these images has been developed so that they can be 
utilized by the project efficiently. 

Through the standing orders, the project receives black and white positive 
transparencies on 9-1/2 inch film. These arrive now about one month after 
acquisition. The images may contain a portion or an entire test site, and 
are the first indication that a test site has been acquired on a specific 
date. The standard catalogs arrive later and the microfilm catalog, which is 
very useful, is further delayed. Upon receipt of the transparencies, an 
overlay is produced photographically. The overlay contains all the annota- 
tions and coordinate marks of the MSS 5 (Red Band) image. Geographic coor- 
dinates and the test site coordinates are plotted on the overlay. Using the 
overlay, it is possible to determine the percentage coverage of the test 
site. At this point, a decision is made whether to submit a request for 
digital tapes. The decision is based on cloud cover, haze, percentage of 
test site photographed, as well as on previous acquisitions of the test site. 
The satellite produces a lot of images so that a selection is possible. 

The ERTS images are also being utilized for the photointerpretation tasks 
(section 3) and for the development of terrain spatial signatures (section 5) . 
Both of these operations require changes in scale which are accomplished very 
efficiently photographically. The photointerpretation tasks require enlarge- 
ments while the spatial signatures require high resolutions reductions in 
scale in order to obtain large diffraction patterns. 

2.2 ERTS Digital Tapes 

The first digital tapes were delivered on October 30, 1972, and were 
9-track. Existing digital tape recorders in Itek's Computation Center can 
handle 7-track 556 BPI tapes, so the 9- track tapes could not be used without 
converting them to 7-jtrack 556 BPI tapes. The conversion could be done by 
an independent computer center. However, the project would have had to ab- 
sorb the costs of the conversion, additional tapes and the labor associated 
with the handling required by the conversion. It was decided not to burden 
the project with the cost of an unnecessary data handling operation and the 
9-track tapes were returned to NDPF. The project technical monitor, 

Mr. Edmund F. Szajna, was notified of the situation and acted promptly to 
correct it. In the first two weeks of November, a number of tapes were 
delivered which were 7-track, 800 BPI. These could not be used also and 
were returned. Finally, in the end of November, several 7-track, 556 BPI 
tapes were delivered and the project is utilizing them in several digital 
processing operations. 
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2.3 Aircraft Images 

Low altitude photography (10,000 feet) from all six test sites has been 
delivered by NASA-MSC. High altitude photography (60,000 feet) from two 
test sites, Weslaco, Texas and New Orleans, Louisiana, has also been delivered. 
In general, the photography is very good and is being fully utilized by the 
photointerpretation tasks (section 3). It has been observed that the high 
altitude photography is very useful because of the larger area covered: 

14 x 50 square nautical miles versus 2.8 x 50 square nautical miles for the 
low altitude photography. It is highly desirable to obtain high altitude 
photography either by a U-2 or RB57F aircraft over the remaining four test 
sites. 

3. PHOTOINTERPRETATION TASKS 

Photointerpretation of the ERTS and aircraft images from the six test 
sites is being conducted for two reasons: 

1. Ground truth identified on a map must be located accurately in 
the ERTS images. The ground truth location is accomplished 
through the use of larger scale images of the same area from 
aircraft underflights. The technique had to be tried and re- 
fined for ERTS-A images. 

2. A lot of detail is obvious in the ERTS images and most of it 

can be identified by comparison to maps and aircraft photography. 

The objects or structures that can be identified give clues as 
to the information that can be retrieved digitally from the 
ERTS images. 

A photointerpretation task on the Weslaco, Texas test site (ERTS image 
1038-16314) has been completed and another task on the Cascade Mountains test 
site is in progress. The conclusions drawn from the photointerpretation of 
the Weslaco area are: 

1. The ERTS images have sufficient resolution for monitoring agricul- 
tural resources in crops larger than 40 acres. This conclusion 
basically agrees with Professor Colwell's findings for ERTS images 
of California. 

2. Substantial cloud cover (about 50%) as is experienced during 
summer over Weslaco, greatly reduces the usefulness of a multi- 
spectral analysis of farms since most farms of 160 acres or 
larger are completely or partially covered by clouds or their 
shadows . 

3. The IR2 band (0.8 - 1.1 micron) gives excellent results for the 
identification of surface water. Even small bodies of water 
(about 4 acres) were definitely identified and easily distinguished 
from cloud shadows. This band does not appear to give information 
about water depth or clarity, but defines rather precisely the 
boundaries between wet areas and dry land. 

4. Most geographic features such as lagoons, lakes, islands, river 
beds, marshes, large canals, docks can be clearly identified in 
the ERTS images. 


3 



5. The ERTS images provide information for the analysis of river 
effluents, silting of harbors and the movement of water with 
the tides. 

6. Urban areas are not well identified from spatial characteristics, 
but can be delineated from multi-spectral signatures. 

7. Major highways can be identified, but smaller roads are not 
visible. 

8. Haze appears to have reduced the contrast of the ERTS images even 
though the red, IR1 and IR2 bands are able to penetrate haze. 

9. Very small bright objects such as gas tanks or buildings can be 
detected in the ERTS images and can be identified through the aid 
of accurate maps . 

10. There is an apparent need to update existing maps (1:250,000 scale) 
of the area. 

11. The red band (0.6 - 0.7 micron) has the most information. It 
carries some of the information available in the other three bands. 

12. In the ERTS images, there is no evidence of variation in reflectance 
due to the look angle, while in the aircraft images, substantial 
variation due to look angle and specular reflection from bodies of 
water was readily apparent. 

4. DIGITAL INTERPRETATION TECHNIQUES 

4.1 Multi-Spectral Recognition Program 

The multispectral recognition program identifies resources by their re- 
flectances in several spectral bands. The algorithm assigns a picture element 
to one of a number of resource classes using the maximum likelihood ratio (see 
Reference 1) . The number of classes and their covariance matrices are a 
priori determined. 

The program consists of two parts; one for training operations and another 
for recognition operations. For training, one or more multi-spectral images 
of known classification can be inputed and the training program will compute 
the covariance matrices for the classes present in the data. Conversely, the 
covariance matrices can then be employed to assign new data to resource 
classes by the maximum likelihood ratio criterion which minimizes the proba- 
bility of error. The technique has been the classical approach in pattern 
recognition and can be applied to many statistical recognition problems by 
replacing the spectral reflectance data with measurements of other physical 
quantities. The technique is limited to recognizing resource classes for 
which the covariance matrices are known (from training data) . Also, the pro- 
bability of error increases if the distribution of spectral data for a given 
class deviates significantly from the Gaussian multivariance distribution 
associated with the covariance matrix. 

A number of errors had been noted in the multi-spectral recognition pro- 
gram and a number of tests were done with simulated MSS data to find the 
errors and check the operation of this program. Simulated data was used in 
order to be able to check closely the various program functions. The simu- 
lated images were generated from published (see Reference 2) reflectance 
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values of 16 different materials at specific wavelengths. Average reflec- 
tances for these materials over the MSS spectral bands were computed. Four 
spectral images (one for each spectral band) were developed for training 
purposes and four different images for recognition. Each image, contained 
1,024 x 1,024 pixels and it was divided into 16 squares (one for each class) 
of 256 x 256 pixels. Each pixel within a square was computed from a Gaussian 
random number generator such that the mean was equal to the average reflec- 
tance of the specific material in the particular spectral band. The standard 
deviation was selected randomly for each class and varied between 2% and 50% 
of the mean value. With a Gaussian number generator, some of the pixels came 
out negative and were replaced by zeroes. Also, due to the random number 
generator, all pixels were statistically independent of each other. 

One set of spectral images was employed for training and the covariance 
matrices were computed by the program and checked manually for accuracy. The 
next set of spectral images was then used for testing the recognition algor- 
ithm, which employed the covariance matrices computed from the training 
operation. The program recognized correctly 94.5% of the pixels and rejected 
(did not classify) 0.8% of the pixels that had a zero value at one or more 
spectral bands. During the test, all errors in the program were found and 
corrected. 

This program has not been tested with ERTS images due to the unavailabi- 
lity of 7-track, 556 BPI tapes. Such tapes are available now. A separate 
program to unpack the MSS bulk tapes has been written and successfully tested 
by retrieving the red MSS image and recording it on photographic film through 
our laser beam recorder. Under development is a program which determines the 
number of classes and their covariance matrices from an unknown set of data. 
This program will eliminate the need for training data or images. 

4.2 Edge and Boundary Detection Software 

In the analysis of MSS multi-spectral images, it is important to identify 
the regions which have the same characteristics and to develop boundaries be- 
tween adjacent regions. Theoretically, if each pixel is spectrally identified, 
then boundaries around image portions with the same characteristics can be 
developed. This approach has the drawbacks that the processing is inefficient 
in terms of computer time and normal errors in the multi-spectral detection 
create erroneous boundaries. In addition, no advantage is taken of any spa- 
tial information present in the images. 

A new technique has been developed to perform automatic terrain classifi- 
cation based on edge detection of terrain regions characterized by spatial 
signatures. Boundaries of the regions are developed by derivative operations 
on digitized imagery. The derivatives alone do not lead to satisfactory boun- 
daries. So, the boundary development starts with a low resolution version of 
the image (25X reduction), obtained by averaging groups of 25 x 25 pixels. 

This image is subjected to a derivative operator and a threshold to determine 
the location and direction of candidate edge elements. Topographic constraints 
are then imposed: the boundaries must be continuous and end on intersections. 
This operation eliminates isolated edge elements and fills the gaps in piece- 
wise continuous boundaries. The end result is the partitioning of the image 
into regions. 
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This boundary map is then employed as a plan for the boundary map of the 
next higher resolution image, which is reduced only five times in relation to 
the original image. The 25X reduction boundary map is enlarged five times to 
fit the 5X reduction image. Then, the boundaries become at least 5 pixels 
wide. The boundaries of the intermediate image (5X reduction) are constrained 
to lie within the enlarged 25X reduction boundary plan. The intermediate 
image is subjected to a derivative operator and the resulting image is set to 
zero, except where it coincides with the enlarged boundary map. This opera- 
tion defines the regions where the intermediate image boundaries are allowed 
to exist. In these regions, the intermediate derivative image is shrunk in 
width until it is only one pixel wide, the pixel being located on the deri- 
vative centroid. The shrinking operation guarantees boundary continuity and 
the boundaries are now only one pixel wide. An undesirable by-product of the 
operation is branches or appendages connected to the real boundaries. These 
are subjected to a topographic constraint and they are eliminated if they 
don't meet the constraint. 

By a similar operation, the intermediate image boundary map is used as a 
plan for determining the boundaries of the high resolution original image. 

The end result is the partitioning of the image by the boundaries into regions. 

The technique is elaborate, but avoids the pitfalls of simple derivative 
operations such as discontinuous, noisy, erroneous or multiple boundaries. 

One limitation is that an error in the 25X reduction boundary plan is propa- 
gated with no correction to the final boundary map. To correct errors in the 
first boundary plan, spatial terrain recognition has been employed in small 
regions of the original image selected by a rectangular grid. Changes in the 
terrain classification between adjacent regions indicates a boundary between 
them. No change indicates either no boundary or two boundaries. The results 
of the terrain classification are used to correct the boundary plans. Further- 
more, the terrain enclosed by the boundaries can now be classified, and the 
boundary detection technique has evolved into an accurate regionalized terrain 
classification scheme. As such, the technique described has certain advan- 
tages : 

1. Only 25% of the image is subjected to the spatial pattern recog- 
nition which is time-consuming. 

2. Better classification results are achieved than if Only spatial 
pattern recognition was applied to the entire image. For two 
specific images, the detection rates for cultivated land increased 
to 93.6% and 96.6% from 91.0% and 87%, respectively. 

3. The classified regions are accurately delineated. 

4.3 Digital Image Registration 

In order to digitally compare images acquired on different dates, of 
different spectral bands or different sensors, it is important to register 
them digitally. The project is employing MSS system corrected (bulk) spectral 
images. The MSS spectral images of a given area acquired on a given date by 
the satellite form a set which is inherently registered. However, images from 
different dates are not registered. In this case, to perform change detection 
one must register at least two spectral images from two different dates. This 
can best be done by registering the two red band MSS images to each other. 
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The registration technique depends on correlation of the two images . To 
reduce the required computer processing time, only a number (10-20) of small 
(64 x 64 pixels) areas are correlated. The shift in the correlation peak 
between two associated areas indicates the direction and magnitude of the 
shifting required to bring the two areas in registration. The areas that are 
correlated form the control points which provide samples of the relative dis- 
tortion and displacement of the two images. To bring the two images in regis- 
tration, every pixel is shifted by an amount which is a linear interpolation 
of the required shifts for the nearest control points. By drawing imaginary 
lines between control points, the images can be divided into triangular non- 
overlapping regions. A pixel located within one triangular region is sub- 
jected to x-y shifts which are linear interpolations of the required shifts 
for the three control points located at the apeces of the triangle. This 
method works well and insures continuity in the shifted images. 

The accuracy in the image registration is affected by errors in the 
correlations of the control points and to the extent that the relative image 
distortion deviates from the linear interpolation. The second type of error 
is easily corrected by increasing slightly the number of control points. How- 
ever, errors in the correlations of the control points are the most trouble- 
some. These arise from several sources: 

1 . Changes in the image such as cloud cover, or seasonal changes . 

This problem is avoided by selecting the control points manually 
and by correlating the image derivatives rather than the images 
directly. 

2. Since the areas around the control points which are used for 
correlation are limited in size (64 x 64 pixels) the correlation 
may produce wrong answers when the displacement of two associated 
areas is a substantial portion of the area size. It has been 
observed that for good correlation accuracy, the image detail 
between two areas being correlated should overlap by at least 
80%. Stated another way, the actual displacement between two 
areas correlated should be less than 20% of the area size. 

This effect has been known as the correlation pull-in range 
and has also been observed in electronic image correlators. 

The problem of the limited pull-in range has been solved by a three step 
correlation technique. In the first step, the images are approximately re- 
gistered manually. In the second step, the areas to be correlated around the 
control points are 320 x 320 pixels and are reduced (5X) by averaging to 
64 x 64 pixels. The reduced control point areas are correlated and the 
original size images are shifted by linear interpolation. After this step, 
the images are coarsely registered within 10 pixels. In the final step, the 
control point areas are again 64 x 64 pixels, but in the original image reso- 
lution (no 5X reduction) . The correlations of the control points are carried 
out again and the images are shifted by linear interpolation. After this step 
the images are registered within two pixels. 

As mentioned earlier, two control areas are correlated after they have 
been subjected to a derivative operation, and all negative values have been 
inverted (made positive). The derivative operator is conveniently applied as 
a Fourier plane filter. For example, the Laplacian operator is equivalent to 
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multiplying the Fourier transform of the image by the function (w x 2 + Wy^) 
where w x and Wy are the frequency variables in the x and y directions. We 
have tried both the gradient and Laplacian operators on two Apollo 9 photo- 
graphic images (ERTS digital tapes were not available until recently) of the 
Salton Sea - Imperial Valley area. The images were the red band and the IR 
band, and the IR image was successfully registered to the red band. The 
Laplacian operator gave correlation functions with sharper peaks. 

5. DEVELOPMENT OF TERRAIN SPATIAL SIGNATURES 

The multi-spectral recognition techniques, described in section 4.1, 
ignore the spatial information in an image. Yet, the information conveyed 
to a human observer by a black and white image is purely spatial. The auto- 
mated classification of resources can be significantly improved and expanded 
by utilizing the spatial information in the images. Terrain types can be 
digitally recognized if they can be associated with spatial signatures which 
can be defined as unique patterns in a set of measurements involving a small 
area of an image. The area size is bounded on the low end by the image re- 
solution and on the high end by image characteristics. In other words, the 
average area analyzed should be significantly larger than image resolution 
(at least 10 x 10 pixels) and should not be so large as to include more than 
one type of terrain. Of course, the second condition cannot be met for every 
conceivable area in the image because an area selected around the boundary of 
two terrain types will by necessity display both types of terrain (such as a 
lake-cultivated land boundary). Fortunately, as discussed in section 4.2, it 
is possible to recognize the terrain types of an image by subjecting only 25% 
of the image area to spatial terrain signature analysis and the areas tested 
are selected so they don't contain boundaries. The area sizes that we con- 
veniently use are 32 x 32 and 64 x 64 pixels. 

The measurements that may be made and the technique for extracting a spa- 
tial signature are not restricted. The Fourier transform of an image is a 
convenient means to isolate and extract spatial signatures. The reason is 
that repetitive image characteristics are transformed into discreet frequen- 
cies which can be easier isolated. 

The spatial signatures that can be isolated are dependent on image 
characteristics such as resolution and scale. For the ERTS images, spatial 
signatures are determined by examining the Fourier transforms from both ERTS 
and aircraft images for the same geographical area. The aircraft images being 
of higher resolution are useful for identifying spatial signatures which are 
very weak in the Fourier transforms of the ERTS images (such as urban areas) . 
The most efficient way to establish terrain signatures for the ERTS images is 
optically, using the Fourier transforming properties of lenses. We are em- 
ploying a special optical bench for this purpose which permits photographing 
the diffraction patterns and the images simultaneously. The bench also allows 
photometric measurements of any part (rings or wedges) of the diffraction 
patterns. 

The relationship between Fourier transforms and diffraction patterns is 
not exact, the diffraction pattern being the amplitude squared of the Fourier 
transform of the square root of the image. If the image is photographically 
developed to a gamma of two, then the diffraction pattern is the amplitude 
squared of the Fourier transform of the image. 


3 


8 



To obtain good size diffraction patterns, a long focal length lens (48 
inches long) is being used. The diffraction patterns have some artifacts 
not related to the terrain images. These are: 

1. Rings which are due to the circular aperture employed to limit 
the image area being transformed. These rings are also present 
in a digitally computed Fourier transform. 

2. Two frequency spots due to the line structure present in the ERTS 
images. The line structure is a characteristic of the multi- 
spectral scanner and electron beam recorder system. 

3. Artifacts due to the optical bench such as lens aberrations, 
laser beam non-uniformity, and phase modulation by the trans- 
parencies, since a liquid gate containing refractive index 
matching fluid is not used. The liquid gate provides cleaner 
diffraction patterns by inserting the transparencies into the 
gate, but the additional handling of the transparencies is in- 
convenient. 

To eliminate or suppress artifacts in the diffraction patterns, a mask 
is employed when photographing the diffraction patterns. The mask is itself 
the photograph of the diffraction pattern of an image area (from ERTS images) 
with no detail such as areas of water from lakes or the ocean. 

The masks have been used with very good results. Diffraction patterns 
photographed through them, display more clearly the spatial signatures asso- 
ciated with the various types of terrain, while artifacts have been substan- 
tially suppressed. The diffraction patterns of terrain types from eight ERTS 
images have been obtained, and are being analyzed to establish the uniqueness 
of spatial signatures. The terrain types include: mountains, rivers, urban 

areas, water, desert and cultivated land. Cultivated land from ERTS images 
has displayed unmistakably the familiar four frequency peak signatures we had 
isolated in 1971 in Apollo 9 photography of the Imperial Valley. 

The final step in exploiting the spatial signatures isolated will be to 
write algorithms which detect the spatial signatures digitally. These will be 
incorporated into the existing Pattern and Terrain Classification Software 
System. 

6. CONCLUSIONS 

From the work done during the reporting period, we can draw certain con- 
clusions : 

1. It is possible to register digital images even from different 
spectral bands. 

2. It is possible to develop boundaries of resources and improve the 
terrain recognition accuracy. 

3. It is possible to merge the multispectral and terrain recognition 
results and expand the resource classes that can be recognized. 

4. Spatial pattern signatures can be developed from the ERTS-1 images 
for recognizing terrain types. 
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5. Photointerpretation results have shown that most geographical 
features are recognizable in the ERTS-1 images. 

7. PROGRAM FOR NEXT REPORTING PERIOD 

The work planned for the remainder of the investigation involves the 
computer processing of ERTS images to determine the degree of success of the 
interpretation techniques and the resources that can be reliably recognized. 

ERTS-1 images from the six test sites will be processed through the 
developed software. 

8. RECOMMENDATIONS 

High altitude (60,000 feet) aircraft photography over four test sites 
would be very useful to the investigation. 

It is recommended that for better results the techniques developed be 
implemented with substantial human interaction. A completely automated 
approach lacks flexibility and does not seem to be workable. Since the 
processing is done digitally, computer driven displays to provide quick 
man-computer interaction are highly desirable. 
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